Statistical Modeling and Conceptualization of Visual Patterns

Natural images contain an overwhelming number of visual patterns generated by diverse stochastic processes. Defining and modeling these patterns is of fundamental importance for generic vision tasks, such as perceptual organization, segmentation, and recognition.Theobjectiveof this epistemological paper is to summarize various threadsof research in the literature and to pursueaunified framework for conceptualization, modeling, learning, and computing visual patterns. This paper starts with reviewing four research streams: 1) the study of image statistics, 2) the analysis of image components, 3) the grouping of image elements, and 4) themodeling of visual patterns. The models from these research streams are then divided into four categories according to their semantic structures: 1) descriptivemodels, i.e., Markov random fields (MRF) or Gibbs, 2) variants of descriptivemodels (causal MRF and “pseudodescriptive” models), 3) generativemodels, and 4) discriminativemodels. The objectives, principles, theories, and typicalmodels are reviewed in each category and the relationships between the four types of models are studied. Two central themes emerge from the relationship studies. 1) In representation, the integration of descriptive and generative models is the future direction for statistical modeling and should lead to richer andmore advanced classes of visionmodels. 2) Tomake visual models computationally tractable, discriminative models are used as computational heuristics for inferring generativemodels. Thus, the roles of four types ofmodels are clarified. The paper also addresses the issue of conceptualizing visual patterns and their components (vocabularies) from the perspective of statistical mechanics. Under this unified framework, a visual pattern is equalized to a statistical ensemble, and, furthermore, statistical models for various visual patterns form a “continuous” spectrum in the sense that they belong to a series of nested probability families in the space of attributed graphs.